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Abstract 

Background: Quality improvement (Ql) methods are widely used in surgery in an effort to improve care, often 
using techniques such as Plan-Do-Study-Act cycles to implement specific interventions. Explicit definition of both 
the Ql method and quality intervention is necessary to enable the accurate replication of effective interventions in 
practice, facilitate cumulative learning, reduce research waste and optimise benefits to patients. This systematic 
review aims to assess quality of reporting of Ql methods and quality interventions in perioperative care. 

Methods: Studies reporting on quality interventions implemented in perioperative care settings will be identified. 
Searches will be conducted in the Ovid SP version of Medline, Scopus, the Cochrane Central Register of Controlled 
Trials, the Cochrane Effective Practice and Organisation of Care database and the related articles function of 
PubMed. The journal BMJ Quality will be searched separately. Search strategy terms will relate to (i) surgery, (ii) Ql 
and (iii) evaluation methods. Explicit exclusion and inclusion criteria will be applied. Data from studies will be 
extracted using a data extraction form. The Template for Intervention Description and Replication (TIDieR) checklist 
will be used to evaluate quality of reporting, together with additional items aimed at assessing Ql methods 
specifically. 

Systematic review registration: PROSPERO CRD4201 401 2845 
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Background 

Quality Improvement (Ql) methods are specially de- 
signed efforts and processes aimed at generating im- 
provements in patient care [1]. Such methods include 
those based on Lean, Six Sigma, Plan-Do-Study- Act 
(PDSA) cycles, Total Quality Management and Continu- 
ous Quality Management, audit and feedback, and many 
others [2]. Guidance on reporting of Ql studies [3] and 
of intervention delivery in evaluative studies [4-7] has 
been published. Surgery is an especially important area 
for quality improvement: an estimated 234 million sur- 
gical interventions are performed every year worldwide 
[8], yet it remains hazardous and prone to error and 
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complication. An international drive to improve quality 
of care in surgery is now supported by initiatives such as 
the Centre for Global Surgery [9] and the Lancet Com- 
mission for Global Surgery [10]. Yet the quality of report- 
ing of interventions in Ql studies in surgery is unknown. 
This is an important problem, as it is increasingly re- 
cognised that explicit descriptions of interventions are ne- 
cessary to ensure that successful interventions can be 
replicated in practice, to avoid research waste, to facilitate 
cumulate learning and to ensure that patients gain the 
best possible benefits from any learning from Ql studies 
[11,12]. We seek to adopt and adapt the Template for 
Intervention Description and Replication (TIDieR) check- 
list [4] to evaluate the quality and completeness of repor- 
ting of studies of quality improvement interventions in 
perioperative care. 
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Perioperative care is a process encompassing care re- 
ceived before, during and after a surgical procedure [13]. 
The translation of successful QI strategies into surgical 
practice has the potential to contribute towards ensuring 
the delivery of safe, high-quality, accessible and affordable 
surgery [2,14-18]. Systematic review has evaluated data 
generated by QI methods across cardiothoracic, colorectal 
[14], vascular, hepatobiliary and upper gastrointestinal spe- 
cialties [2,15,16] and has reported measureable improve- 
ments across the whole perioperative journey including the 
preoperative period (reduction in time to surgery [16]), 
intraoperative period (reduction of sepsis [16]) and post- 
operative period (reduction of surgical site infection 
[14,16], central venous catheter infection [16] and venous 
thromboembolism [16]) Yet the QI literature in surgery 
has also been found to suffer from a range of problems 
including lack of explicit rationale, poor detail and overlap- 
ping components in the published descriptions of QI 
methods [2,19] and quality interventions [20]. The extent 
and quality of patient and public involvement (PPI) in sur- 
gical research are also unclear, despite recommendations 
that the Guidance for Reporting Involvement of Patients 
and Public (GRIPP) checklist be used in order to provide a 
quality assurance on the level of PPI reporting [21]. 

One problem in assessing the literature on quality im- 
provement is a degree of conceptual and terminological 
confusion over the term 'intervention'. The methods of 
improvement are sometimes referred to as interventions, 
yet so too are the quality interventions that such me- 
thods seek to implement. Thus, for example, the litera- 
ture may use the term 'intervention' interchangeably to 
describe both application of the PDSA method and a 
quality intervention such as a checklist or 'bundle'. 

For purposes of this review, QI methods will be de- 
fined as the processes (such as PDSA cycles) which are 
typically intended to support the implementation of a 
quality intervention. Quality interventions will be defined 
as the individual components of care delivery which are 
implemented in order to achieve an improvement in the 
delivery of patient care. Quality interventions need to be 
described explicitly and precisely if it is to be possible to 
implement them. The parameters that might be used in 
describing such interventions include 

• What (which materials, and activities should 
be used) 

• Who (qualification type and competency) 

• How (face-to-face, in a group, via the internet) 

• Where (setting, infrastructure) 

• When and how much (dose, timing, frequency, 
duration) 

• Tailoring (personalisation) 

• Modifications (changes during the course of 
the study) 



• How well (what challenges were identified, e.g. 
dropouts or missing data) 

Aims 

This systematic review aims to assess the completeness 
of reporting within the perioperative literature on QI 
methods and quality interventions and to identify which 
elements are most frequently missing. 

Methods 

Design 

We will undertake a review of the published qualitative 
and quantitative surgical literature on QI. We will define 
QI methods as the processes which are usually intended 
to support implementation of the quality intervention 
such as PDSA cycles. We will define quality inter- 
ventions as the individual components of care delivery 
which are selected in order to make an improvement 
(such as issuing checklists or care bundles). 

Eligibility criteria 

This review will include 

• All studies published between 1 January 2000 and 
28 May 2014 to capture all papers indexed since the 
publication of the Institute of Medicine's 'To Err is 
Human: Building a Safer Health System' report [22], 
which highlighted the importance of systems-based 
interventions to address quality and safety problems 

• All surgical specialities 

• Adult surgical services 

• Elective and emergency (trauma) surgery 

• Primary and secondary care, because hospital stay is 
just one aspect of the surgical patient's whole 
pathway [23] 

• Studies using both validated and unvalidated 
measures 

• Studies meeting the criteria within the QI taxonomy 
generated by Shojania et al. [24] (Table 1) 

• All epubs ahead of print which are indexed in one of 
the selected databases by the end date specified for 
the review 

• QI papers reporting upon a deliberate effort to 
produce change in perioperative care. This may be 
in the form of a QI report, or a study of a QI 
method or quality intervention 

This review will exclude 

• Audits, unless they explicitly report on the 
implementation of a QI method which is designed 
to produce and evaluate a change 

• Qualitative papers reporting exclusively on staff or 
patient experience of using QI methods 



Jones et al. Systematic Reviews 2014, 3:98 
http://www.systematicreviewsjournal.eom/content/3/1/98 



Page 3 of 6 



Table 1 Quality improvement taxonomy 

Ql strategy Definition Methods 

1. Provider education Dissemination of information Educational outreach visits 

Distribution of educational 
material 



2. Provider reminder systems 

3. Patient reminders 

4. Promotion of 
self-management 

5. Audit and feedback 

6. Patient education 

7. Organizational change 

8. Financial, regulatory, or 
legislative incentives 

9. Facilitated relay of clinical data 
to providers 



Any 'clinical encounter-specific' 
information intended to prompt 
a clinician to recall information 
or consider a specific process 
of care 

Any methods of encouraging 
patient compliance to 
self-management 

Access to a resource that 
enhances the patients' ability to 
manage their condition 

Any feedback of clinical 
performance 



Dissemination of information 



Any change in organizational 
structure 



Any financial bonus, 
reimbursement or provider 
licensure scheme 

Transfer of clinical information 
from patients to the provider 
when data was not collected 
during a patient visit 



Decision aids 
Reminders 

Appointment reminders 

BP devices 

Fit Bits/pedometers 

PROMs 
LOS 

Morbidity and mortality 

Distribution of educational 
material 

Individual or group sessions 
Multidisciplinary teams 
Communication 
Health records 

Positive or negative incentives 
for providers or patients 

Telephone call 

Postal contact 



Surgical examples 

Component separation training 
and recurrence rates 

Cadaveric training and surgeon 
confidence 

MEWS 

The WHO surgical safety checklist 



SMS exercise reminders before 
bariatric surgery 

Follow up phone calls with 
recommended adjustments to 
care 

Percentage of patients achieving 
target LOS 



Tri-modal pre-habilitation 
programme compliance and 
effect on LOS 

Changes to staff rota to facilitate 
early patient mobilization after 
elective arthroplasty 

1 8-week wait target for elective 
orthopaedic surgery 

Relay of BP measurements to the 
pre-assessment team 

Collection of postoperative 
complication data through postal 
survey 



Adapted from Shojania et al. [24] Closing the Quality Gap: A Critical Analysis of Quality Improvement Strategies (Vol. 2: Diabetes Mellitus Care). Technical Reviews, 
Rockville (MD): Agency for Healthcare Research and Quality (US). LOS length of stay, MEWS Modified Early Warning System, BP blood pressure, WHO World Health 
Organization, SMS Short Message Service. 



• Papers reporting on screening programmes and 
diagnostic interventions such as endoscopy and 
end-of-life care 

• Papers reporting on secondary analyses where the 
main results have been published elsewhere 

• Editorials and articles not published in the English 
language 

• Abstracts and conference proceedings 

Disagreements about eligibility will be resolved by dis- 
cussion within the team. 

Search strategy 

Databases will be selected for their ability to represent 
surgical and improvement method literature. Searches 
will be performed in the Ovid SP version of Medline, 
Scopus, the Cochrane Central Register of Controlled 



Trials, the Cochrane Effective Practice and Organisation 
of Care (EPOC) database (which indexes interventional 
studies focused on improvement in healthcare delivery) 
and the related articles function of PubMed. The journal 
BMJ Quality will be searched online using the find func- 
tion for perioperative and surgical terms. MeSH terms, 
search terms, thesaurus mapping and Boolean operators 
will be used. 

The search strategy (Figure 1) was designed by three 
reviewers (ELJ, MDW and GPM). It will be conducted 
by an experienced research fellow (ELJ) who will apply 
the restrictions of publication year (2000-2014), humans 
NOT animals, and NOT infants. The search strategy is 
intended to capture terms relating to (i) surgery, (ii) 
quality improvement and (iii) methodology. Improve- 
ment terms were adapted from the improvement science 
research scan produced by the Health Foundation [1]. 
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Combine box A and boxB using AND 



BOX A 

Deming.ti,ab, 6 sigma.mp. , (Six adjl sigma).mp Lean adjl sigma).mp. , Measurement for 
improvement,ti,ab, (quality adj2 improv$).ti,ab., (quality adjl management).ti,ab., {improv$ adj2 
science).ti,ab., {process ad]2 improv$).ti,ab, (Plan and do and study}.ti,ab., PDCA.ti,ab, pdsa.tt,ab,, plan do 
check.ti,ab, (method adj2 improv$).ti,ab r health. mp. and behavioural economics.ti.ab, operations 
research.ti.ab, decision science.ti,ab, Shewhart.ti,ab., Pareto chart.ti.Ab, Statistical process control. mp., 
Statistical quality control. mp, Toyota. mp., Paretochart$.mp, Control chart. mp. 



BOX B 

preoperative. mp„ Per-operative.mp„ surg$.tr,ab, exp General Surgery/, exp Surgical Procedures, Operative/ 
or Operative Time/ , Peri-operative.ti,ab., Perioperative.tt.ab., Pre-operative.ti,ab, Operative.ti,ab. 
CancerS.mp., enhanced re covery.tr, a b., eras.ti,ab„ rapid recovery.ti.ab., fast.mp. and track.ti.ab., Operating 
theatre. mp, Operating room.mp, Operating room$.mp, Anesthe$.mp, Anaesthe$.mp, Trauma. mp. 



Combine using AND to either BOX Cor B0XD 



BOX C: Quantitative terms 

Radnomized.ti,ab, Randomi?ed controlled trial.pt., Randomized controlled trial.pt, controlled clinical 
trial.pt., Randomized controlled trials. sh., random allocation. sh., double blind method. sh., single-blind 
method. sh., Single?blind method. sh, clinical trial.pt., exp clinical trial/, {clinic$ adj25 trial$}.ti,ab., ((singl$ or 
doubl$ or trebl$ or tripl$) adj25 (blind$ or mask$)).ti,ab., placebos.sh., placebo$.ti,ab., comparative 
study.sh., exp evaluation studies/, prospective studies. sh., (control$ or prospective$ orvolunteer$ or 
retrospective$).ti,ab., interrupted time?series, Time-sertes.tt,ab, Repeated measure$ .mp. , Cohort.mp., 
"Case?control".mp., "case control".mp. , (assess$ adj3 process quality). mp, {evaluat$ adj3 process 
quality}. mp., compliance.ti,ab., quality control. mp, Process control .ti,ab, Control chart* .ti,ab. Adherence 
OR adherance.ti,ab, group adj3 compar*.rnp, Control.ti,ab, Before adj2 after.mp., eval.$.ti,ab, Variability OR 
variation OR variable. mp., Checklist$.mp. 



BOX D: Qualitative terms 

Qualitative.ti,ab. , Focus Groups. mp. or exp Focus Groups/ , exp Interview/ or interview.mp. , Interviews as 
Topic.mp. or exp Interviews as Topic/, ethnograph$.mp. , content analysis. mp. , grounded theory.mp, 
grounded approach. mp. , exp Qualitative Research/ or qualitative. mp. , Phenomenolog$.mp. , discourse 
analysis.mp. , constant comparison. mp., observational method. mp , theoretical sampl$.mp. , thematic 
analys?s.mp. , improvement report$.mp. 



Figure 1 Search strategy. 



A training exercise was undertaken with a sample 
of search results where two authors (ELJ and MDW) 
considered selected full-text articles and discrepancies 
were resolved with a third reviewer (GPM). This enabled 
the authors to refine inclusion and exclusion criteria, 
ensuring consensus and reliable article selection. 

A further training exercise will be undertaken whereby 
two authors (ELJ and NJL) will independently rate a sample 
of full-text articles against the TIDieR checklist until a high 
agreement is reached. TIDieR [4] is recommended by the 
Enhancing the Quality and Transparency of Health Re- 
search (EQUATOR) Network as an extension of the Con- 
solidated Standards of Reporting Trials (CONSORT) [25] 
and SPIRIT [26] statements to improve reporting across all 
evaluative' study designs. Each item in the checklist will 
have an explanatory statement to guide the rater on how it 
should be interpreted. In accordance with previous work 
conducted by Hoffman and colleagues [12], the 12 items on 
the TIDieR checklist will be rated as 'Yes' (indicating that 
the description of that element of the intervention had been 
explicit) or 'No' (not reported or not clearly described). 



Two reviewers (ELJ and NJL) will independently assess 
titles and abstracts of all abstracts to select and obtain 
full-text articles. The search results will be supplemented 
with hand searching of the reference lists of the full-text 
articles and of one recently published systematic review 
on improvement science [2] (ELJ). 

Data extraction 

This review defines a quality intervention as a change to 
process directed at securing improvement, for example, 
introducing joint working patterns at weekends for 
assistant practitioners and foundation year 1 doctors to 
improve the rate of peripheral cannula insertion to re- 
duce missed antibiotics. A QI method is defined as the 
process by which the change is supported and facilitated, 
for example, PDSA cycles. Patient and public involve- 
ment will be defined as the incorporation of the know- 
ledge, skills and experience of patients, carers and the 
public into a study [27,28]. 

Data will be extracted from each selected paper by ELJ 
and NJL using standardised Excel templates. The first 
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template will contain the 12 TIDieR checklist items [4] for 
recording the description of the intervention. The second 
template will contain key elements of the QI method. For 
purposes of this analysis, the salient features of quality 
improvement methods were determined by the authors 
following review and discussion of the relevant literature 
(see Table 2 for the list of features). When reports contain 
descriptions of two interventions, they will be rated 
separately. Checklists have also been published to facilitate 
reporting of PPI [21,29]. However, the emphasis of this 
review is upon the completeness of the QI reporting so 
the papers will be judged against only one PPI criterion, 
scoring 'DONE' if patient involvement is specified, 'NOT 
CLEAR' if it is not reported and 'NOT DONE' if patients 
are explicitly not involved. Discrepancies in the allocation 
of checklist scores will be resolved by discussion with a 
third reviewer (GPM or MDW). 

Data analysis 

Data will be analysed descriptively using an Excel data 
extraction sheet. Nominal data will be used to present 
the proportion of complete and incomplete TIDieR 
checklist items. The potential transferability of findings 
between contexts will be considered. This is a descrip- 
tive review, and meta-analysis will not be undertaken. 

Consistent with the principle that reviews may engage 
an iterative process [30], the review may evolve iteratively 
to include additional analysis such as bibliometric mea- 
sures and descriptions of the fidelity of the interventions. 

Discussion 

This review has a number of strengths and limitations. It 
will be the first review assessing how well QI methods 

Table 2 Data extraction template items 



and quality interventions are described across diverse 
settings (emergency and elective, and primary and se- 
condary care) in perioperative care. This will advance 
understanding on what is required to improve reporting 
on QI methods and quality interventions. The review 
will extend beyond a presentation of raw outcome data, 
also considering the following: What rationales are pro- 
vided for the application of specific QI strategies? How 
is QI defined? To what extent are patients involved? The 
findings of this review will be used to generate a re- 
search protocol to identify and resolve the challenges as- 
sociated with defining and providing accounts of all of 
the elements of QI methods and interventions in sur- 
gery. This knowledge will generate a practical framework 
to facilitate the replication of effective QI strategies in 
practice. This framework will be pilot tested to confirm 
that it is a reliable method of specifying and describing 
the elements of QI methods and quality interventions. 
We anticipate that this work will be relevant to a wide 
multi-disciplinary community of clinicians and researchers 
who wish to reliably accelerate positive changes to prac- 
tice to improve quality of care for patients and to improve 
the quality of the reporting of QI methods and interven- 
tions in perioperative literature. 

Our study may have limitations. Papers not published 
in English will be excluded due to resource limitations, 
which may introduce bias. However, research has sug- 
gested that such exclusions tend to have a limited effect 
overall on systematic review conclusions [31]. Steps have 
been taken to limit potential subjectivity in data analysis 
by including standardised data extraction tools and 
checklists and by achieving consensus with a third re- 
viewer. A team of social scientists and clinicians will 



Demographics 



Quality intervention (TIDieR parameters) 



QI method 



Author, year, country, surgical 
speciality 



1. Brief name 

2. Why (rationale for intervention) 

3. What (materials used to apply the intervention) 

4. Procedures (processes used in the intervention) 

5. Who (who delivered the intervention, including 
level of training) 

6. How (mode of delivery: face to face, internet) 

7. Where (location: emergency or elective, and primary 
or secondary care) 

8. When and how much (duration, dose, intensity) 

9. Tailoring (was the intervention planned to be personalised) 

10. Modifications (describe what, why, when and how 
modifications were made) 

1 1. How well (strategies to improve or maintain 
compliance) 

12. How well (outcome of compliance assessment) 



1. Sample size 

2. Baseline measurement 

3. Data collection schedule 

4. Data analysis (e.g. driver diagrams) 

5. Data volume/duration (e.g. length of 
PDSA cycle) 

6. Explicit description of prediction of change 

7. Missing data (and reasons given) 



8. Description of generalizability 

9. Adverse effects (on health care providers and 
resource utilisation) 

10. Presence and type of patient or stakeholder 
involvement (collaborative or consultative) 
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ensure that key messages most appropriate to a surgical 
audience will be disseminated, addressing gaps in the 
current reporting of QI methods and interventions. 
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